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(57) Abstract: A system for controlling playback of a video program recorded on an optical disc or other recording medium uses an 
electronic data file (100) that is independent of, but corresponds to, the video program recorded on the recording medium (106). The 
electronic data file provides a hierarchical segmentation of the video program and associates supplemental information with various 
segments of the video program. Consequently, the user can access the supplemental information to learn more about that segment 
of the video program or screen that segment for potentially objectionable content. The user may also edit the playlist of segments in 
the electronic data file to control the playback of the video program. 
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TITLE OF THE INVENTION 

Method and System for Controlling and Enhancing the Playback of 
Recorded Audiovisual Programming 

FIELD OF THE INVENTION 

The present invention relates to the field of video recording and playback. 
More specifically, the present invention relates to the field of prepared video 
productions for educational and entertainment use. The present invention 
encompasses a novel software package for controlling and enhancing the playback 
of a pre-recorded video program, where the software package allows the viewer to 
edit the playback of the video program for selected content, access supplemental 
information about the program content and monitor the effect of the editing 
decisions on the playback of the program. 

BACKGROUND OF THE INVENTION 

Television and cinema provide a vast array of available video 
programming. By watching a television broadcast or attending a film showing, a 
viewer can experience almost any type of video programming he or she desires. 
Additionally, in order to make such video programming even more accessible to 
viewers, such video programming may be recorded on such media as video 
cassettes or optical discs for playback using an appropriate recording and playback 
device at the viewer's convenience. 

Being able to play back a recorded video program also provides the viewer 
with greater control over how that video program is viewed. For example, if a 
viewer wishes to replay a portion of the video program, he or she may do so using 
the playback device. Similarly, if the viewer wishes to skip over a portion of the 
video program he or she may fast forward through the program using the playback 
device. 
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If the video programining is on an optical disc, such as a CD-ROM, laser 
disc or Digital Video(Versatile) Disc ("DVD"), the reader of the playback device 
can quickly access any portion of the video program. If the optical disc playback 
device is incorporated into a computer, for example, and controlled by the 
5 computer, the user may specify what portions of the video program and in what 
order those portions should be played by the playback device. 

For example, U.S. Patent No. 5,109,482 to Bohrman discloses a video 
playback and control system in which the viewer may access all the frames in a 
recorded video program, and specify segments of the program by a starting and 
10 ending video frame such that only those segments specified by the viewer are 

replayed. Moreover, the viewer may indicate the order in which segments are to 
be replayed and associate with each segment textual comments relating to the 
segment. 

Similarly, U.S. Patent No. 5,434,678 to Abecassis discloses a system for 
15 controlling the playback of a video program in which segments of the program are 
specified and rated as to potentially objectionable subject matter they contain. 
Using a computer or other user interface device for controlling playback under the 
principles of the Abecassis system, the user specifies a rating for categories of 
potentially objectionable subject matter above which the viewer does not wish to 
20 see rated material. The video program, as recorded, contains alternative sequences 
and video segments which have various ratings of potentially objectionable 
content contained therein. The playback device will then automatically select and 
substitute those segments of the video program which match the preferences 
established by the user for objectionable content in the replay of the video 
25 program. 

While systems such as those disclosed by Bohrman and Abecassis provide 
viewers with a great deal of control over the playback and content of a recorded 
video program, they fail to allow the viewer to monitor the impact such editing has 
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on the video program. Consequently, there is a need in the art for an improved 
video playback control system which allows the viewer great flexibility in 
controlling potentially objectionable content in a video program, but also allows 
the viewer to monitor the impact any editing has on the video program. 

Additionally, the prior art systems generally require that any supplemental 
information or commentary regarding the various scenes of the video program be 
input by the user or recorded on the same medium as the video program. 
Consequently, there is a need in the art for a more flexible way of providing a user 
with supplemental information about the segments and segment content of a 
recorded video program, particularly where no such information was originally 
included on the same recording medium as the video program. 

In connection with these aspects of the art, the prior art systems present the 
recorded video program and any divisions thereof in a flat, linear list. 
Consequently, there is a need in the art for a method and system of better 
organizing a structure associated with a recorded video program to permit more 
rapid and easy navigation by the user among the segments of the program and any 
information associated therewith. 

Other systems related to the use of video programming have addressed the 
desire of viewers to edit out objectionable content. One such device is a program 
for dynamically editing the content of a video program which is broadcast to the 
viewer's television set. In these systems, a signal which indicates the presence 
potentially objectionable subject matter in each segment of the broadcast program 
is included in the broadcast signal in a field which is also used for carrying 
close-captioned text for the video program. The system then monitors this signal 
from the close-captioning field and compares the objectionable nature of the 
program as indicated by the incoming signal with a preset preference provided by 
the user that defines a rating above which programming content should be avoided. 
When the signal from the closed-captioning field of the video broadcast signal 
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indicates material in conflict with the preference setting provided by the user, the 
system may appropriately mute the audio or blank the video output to remove the 
objectionable content from the video program. 

This system, unfortunately, only works if the appropriate rating information 
is provided in the closed-captioning portion of the video programming signal. If 
this is not the case the system will have no ability to edit the programming 
according to the desires of the user. Moreover, this system must simply divide 
potentially objectionable subject matter into relatively broad categories which can 
then be selected or refused by the user. The user has very little control to precisely 
define what he or she may consider objectionable and remove only that material 
from the video program. Finally, the use of video rating information in the 
closed-captioning portion of the video signal requires additional processing of the 
signal by the receiver and playback equipment and prevents the full use of that 
portion of the signal for closed-captioning or other information that may be carried 
there. 

Consequently, there is a need in the art for an improved method and system 
of allowing a user to remove those scenes or audio portions of a video program 
which that user finds objectionable. There is a further need in the art for such a 
system to allow the user great flexibility in determining precisely that subject 
matter which he or she finds objectionable and removing only that subject matter 
from the video program. 

SUMMARY OF THE INVENTION 

The present invention meets the above-described needs and others. 
Additional advantages and novel features of the invention will be set forth in the 
description which follows or may be learned by those skilled in the art through 
reading these materials or practicing the invention. The advantages of the 
invention may be achieved through the means recited in the attached claims. 
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In summary, the present invention may be embodied and described as a 
novel video playback system for playing back a video program recorded on an 
optical disc or other recording medium. This system includes a video playback 
device connected to a reader for reading the video program stored on the recording 
medium, the reader being controlled by the video playback device. The video 
playback device controls playback of the video program in accordance with a user- 
editable electronic data file existing in the video playback device independent of 
data recorded on the recording medium. 

The electronic data file includes a user-editable playlist that specifies a 
division of the segments of the video program as recorded on the recording 
medium. The electronic data file preferably also includes supplemental 
information associated with the specified segments of the video program. 

The video playback device also preferably includes a user interface 
including a display device on which the playlist is displayed. The displayed 
playlist provides a listing of the segments of the video program and may also 
provide a coding system that indicates what types of the supplemental information 
are associated with each segment. The playback device can then access and 
display the supplemental information under control of a user operating the user 
interface. 

Using the user interface, the user can also make changes to the playlist. 
The playback device records the changes to the playlist made by the user through 
the user interface. Thus, the electronic data file may include an original and an 
edited playlist. 

The electronic data file may also provide a description of each of a number 
of audio segments of the video program. The user can edit the playlist to remove 
any of the audio segments from the playlist so that the removed audio segment is 
omitted from a playback of the video program. The playback device also records 
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such deletions of audio segments from the playlist in the edited playlist of the 
electronic data file. 

The system may also preferably generate a statistical analysis of 
characteristics of the video program as played back in accordance with the edited 
playlist. This analysis may be conducted directly on the edited playlist or in 
comparison with an unedited version of the playlist. This analysis will provide 
such data as the running time of the edited program, damage to the plot of the 
program arising from the editing and types of supplemental information associated 
with remaining video segments. 

The present invention also encompasses the methods of making and 
operating the system described above and the method of generating the electronic 
data file described above. For example, the present invention encompasses the 
method of controlling and enhancing playback of a video program recorded on an 
optical disc or other recording medium by controlling playback of the video 
program with a video playback device in accordance with a user-editable 
electronic data file existing in the video playback device independent of data 
recorded on the recording medium. 

The present invention also encompasses the method of providing an 
electronic data file for use in controlling playback of a corresponding video 
program recorded on an optical disc or other recording medium. This method 
includes dividing the video program as recorded on the recording medium into a 
hierarchy of segments; and storing the hierarchy in an electronic data file 
independent of data recorded on the recording medium. This method, consistent 
with the description above, may also include the addition of supplemental 
information stored in the electronic data file and associated with various video 
segments. 

The present invention also encompasses a method of doing business in 
which customers are provided with a means for controlling and supplementing 
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playback of a video program recorded on a recording medium. This method 
includes selling an electronic data file that corresponds to the video program on the 
recording medium, where the electronic data file includes a user-editable playlist 
that divides the video program into video segments and includes supplemental 
information associated with one or more of the video segments, This data file, as 
described above, is usable to control playback of the video program on a video 
playback device. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings illustrate the present invention and are a part 
of the specification. Together with the following description, the drawings 
demonstrate and explain the principles of the present invention. 

Fig. 1 is diagram of a system according to the present invention for 
producing a multi-platform file that supplements and facilitates editing of a pre- 
recorded video program. 

Fig. 2 is a flowchart illustrating the operation of the software of the system 
of Fig. 1. 

Fig. 3 is a diagram of the content of a pre-recorded video program and the 
hierarchical structure of that content according to the multi-platform file of the 
present invention. 

Fig. 3a is a diagram of a second division of the content of a pre-recorded 
video program and the hierarchical structure of that content according to the multi- 
platform file of the present invention. 

Fig. 4 is a diagram of a video program playback system according to the 
present invention. 

Fig. 5 is a diagram of a second embodiment of a video program playback 
system according to the present invention. 
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Fig. 6 is an illustration of the editing system, multi-platform file and 
playback devices of the present invention. 

Fig. 7 is a flowchart illustrating the operation of the software of the 
playback system of Figs. 4-6 according to the present invention. 

Fig. 7a is an illustration of a hierarchical tree of the playlist of the present 
invention. 

Fig. 8 is a diagram of additional divisions of the content of the pre-recorded 
video program for supporting the audio editing features of the present invention. 

Fig. 9 is a flowchart illustrating an additional analysis feature of the 
software of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention provides a method and system for controlling and 
enhancing the playback of recorded audiovisual programming. More specifically, 
the present invention provides systems for generating and using an electronic data 
file that divides the recorded audiovisual program into segments and provides 
supplemental information about the various segments of the program. This 
electronic data file can be used by a number of different playback platforms and 
allows the user to easily edit the playback of the program for appropriate content 
and access supplemental information about program segments. 

Fig. 1 illustrates a system according to the present invention for generating 
the multi-platform data file that corresponds to and supplements a recorded video 
program. This file is referred to herein as an Electronic Film Review or "EFR" 
file. 

As shown in Fig. 1, the system of the present invention for generating an 
EFR (100) preferably includes a computer (108) with an optical disc drive (107). 
As will be understood by those skilled in the art, the principles of the present 
invention can be applied to recorded video programming that is recorded on, for 
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example, a CD-ROM, a laser disc or a Digital Video(or Versatile) Disc ( tc DVD"). 
The term "optical disc" will be used to refer generally to these and any equivalent 
video storage media. 

The invention may also be applied to a recorded video program stored on 
other data storage media including, but not limited to, a magnetic disc, a magneto- 
optical disc, or semiconductor memory. The term "recording medium" will be 
used herein to refer to all such recording media on which a video program can be 
recorded, including, but not limited to, optical discs, magnetic and magneto-optical 
discs, and semiconductor memory unit. 

The computer (108) preferably includes a modem and connection (102) to 
the Internet or World Wide Web (101), Consequently, the EFR maybe sold and 
distributed over the Internet (101). 

The computer (108) also has a monitor (103) on which the video program, 
recorded on the optical disc (106) and retrieved by the optical disc drive (107), is 
shown. With user input devices, such as the keyboard (104) and mouse (105), a 
user can divide the recorded video programming into a hierarchy of segments. 
Additional data can then be associated with each segment which supplements the 
segment, describes the segment or identifies potentially objectionable subject 
matter in the segment. As used herein, a "segment" refers to any specified portion 
of the recorded video program at any hierarchical level, including the entire video 
program. 

The divisions of the video programming into segments and the additional 
information associated with each segment are written to an EFR file (100) which 
exists independently of the programming recorded on the disc (106). The EFR file 
(100) is preferably written in a multi-platform markup language, such as 
Extensible Markup Language (XML) using Unicode to encode the characters. 

The creation of the EFR (100) is done using a video software package 
running on the computer (108). Fig. 2 is a flowchart of this software package and 
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illustrates the method of creating an EFR (100) under the principles of the present 
invention. 

As shown in Fig. 2, the optical disc drive, under control of the video 
software package, reads the data for the video program from the optical disc (201). 
The program, under control of the user, is then displayed on the monitor of the 
computer. The video program consists of a series of frames, still images, that are 
displayed in rapid succession to give the illusion of motion. The user can specify a 
video segment by indicating a starting frame and an ending frame. Each frame 
may be numbered to facilitate selection and identification by a user who is 
defining video segments. 

As the user defines video segments (203), the segment parameters, i.e., an 
identification of the frames that begin, end and are included in the segment are 
written to the EFR file (204). After a segment is identified, the user may provide 
additional data associated with any of the segments (205). This information may 
be background or critical information about the film or the particular segment, 
cultural or linguistic information about the segment, a synopsis of the segment, an 
identification of potentially objectionable content in the segment, a rating of the 
importance of the segment to the plot of the program, etc. Any information the 
user wishes to associate with that segment may be input (205), including numeric 
values or alphanumeric codes that characterize aspects of the segment. The user 
may also designate various types or categories of such additional information. 
This additional data is then written to the EFR (100). All such information, 
whether graphic, textual, animated, etc., is referred to herein as "supplemental 
information." 

This process continues until the user has finished (207), i.e., has defined all 
the video segments desired and entered all the desired supplemental information. 
When the process is finished, the software completes the EFR file (208), which 
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includes the divisions of the video programming and all the supplemental data 
provided by the user. 

Fig. 3 illustrates in more detail the hierarchical division of the video 
program written to the EFR (100). The example illustrated in Fig. 3 is that of a 
DVD (320). When manufactured, the DVD (320) already includes some division 
of the recorded programming into various segments. As shown in Fig. 3, the 
material on a DVD is generally divided into a preview section (302), a main 
segment consisting of the video program (301) and a trailer segment (303) which 
may include out-takes from the video program, director's notes or comments, 
behind the scenes footage, explanations of how the film was made, etc. The video 
program's main segment (301) is usually divided in to a number of chapters (304). 
Consequently, when a user plays a commercial DVD, he or she can choose to 
watch a preview, a trailer or the main program and, within the main program 
(301), can select a particular chapter (304) at which to start watching. Thus, box 
(330) indicates the hierarchical structure provided on a conventional DVD. 

Box (331), however, illustrates additional hierarchical structure and 
divisions of the main program (301) provided in the EFR (100) of the present 
invention. Each of these segments is preferably defined by identifying a first and 
last frame for that segment. The hierarchical structure is for the convenience of 
the user in navigating through the defined segments of the program (301). As 
shown in Box (33 1), each chapter (304) is divided into a number of scenes (305). 
Each scene (305) is divided into a number of sub-scene segments (306). Finally, 
each sub-scene (306) is divided into a number of clips (307). As used herein, the 
term "clips" refers generally to segments of the video program at the bottom level 
of the hierarchy illustrated in Fig. 3. The divisions between the preview (302), 
main segment (301) and trailer (303), and between the chapters (304) in the main 
segment (301) may also be written into the EFR (100). 
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The user may choose to provide supplemental information for any segment 
(304-307) at any hierarchical level in the system (33 1). However, information 
supplementing each clip (307) is preferred and gives the user the most detailed 
information and control over the video program (301). 

Fig. 3a illustrates a second embodiment of the hierarchical structure of the 
segmentation of the video program. According to the principles of the present 
invention, there may be more than one hierarchical structure dividing a recorded 
video program, with all alternate hierarchical structures being stored in the EFR 
(1 00). As shown in Fig. 3a, a chapter (304) may have a first hierarchical structure 
of segments (33 1) as illustrated and described in Fig. 3. The same chapter (304) 
may also have a second hierarchical structure of segments (331a) that divides the 
chapter in a different way. 

One segmentation scheme (e.g., 331) may be best suited to editing the 
video program for presentation in an educational context. The alternative 
segmentation scheme (e.g., 33 la) may be best suited to editing the video program 
for entertainment use. Consequently, two or more alternate segment hierarchies 
may be produced and include in the EFR (100) of the present invention. 

Fig. 4 illustrates a system according to the present invention for using an 
EFR (100) to support and control the playback of the corresponding video 
programming. The exemplary system of Fig. 4 includes a computer (408) which 
includes an optical disc drive (107), e.g., a DVD drive, for playing back video 
programming recorded on an optical disc (106), e.g., a DVD. The video 
programming from the disc (106) is displayed on a monitor (403) of the computer 
(408). As will be understood by those skilled in the art, the computer (408) may 
be, for example, a desk-top computer, a lap-top computer, a server or other 
computer system. The computer (408) is controlled through its user interface, e.g., 
the keyboard (104), mouse (105) and display (103). 
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Under the principles of the present invention, the user of the optical disc 
(106) will also want to acquire an EFR file (100) that corresponds to the 
programming on the optical disc (106), i.e., the EFR (100) delineates a hierarchical 
structure of segments of the video program on the disc (106), such as that shown in 
Fig. 3, and provides supplemental information about various segments of the video 
program recorded on that disc. As will be described in detail below, a video 
playback software package running on the computer (408) will use the data from 
the EFR (100) to enhance the user's enjoyment of the programming on the disc 
(106) and allow the user to better control the playback of that programming. 

The EFR file (100) may be provided to the computer (408) in any of a 
number of ways. As illustrated in Fig. 4, the EFR file (100) may be transmitted to 
the computer (408) via the Internet (101) and connection (1 02). Alternatively, the 
EFR file (100) may be recorded on some medium readable by the computer (408). 
For example, the EFR file (100) may be recorded on a floppy disc (41 1) or CD- 
ROM (410) from which the EFR file (100) can be read into the computer (408). 

Fig. 5 illustrates an alternative playback platform under the principles of 
the present invention. As shown in Fig. 5, the playback device for the optical disc 
(106) may be an intelligent video disc player (500). As used herein, the term 
"intelligent video disc player" refers to a playback device for reading video 
programming from an optical disc, which playback device operates under the 
control of a software package that accepts and utilizes an EFR file (100) to 
enhance the user's enjoyment and control of the playback of the programming on 
the disc (106). 

The intelligent video disc player (500) shown in Fig. 5 includes a video 
disc drive (501) that can read video programming from a video disc and output 
that programming to a connected television set or monitor (550). The player (500) 
also preferably includes a user interface device (505), such as a keypad and display 
on the housing of the player (500), with which the user can control the player 



13 



WO 02/062061 



PCTYUS01/47062 



(500). The user interface device (505) is preferably supplemented by a remote 
control unit (405) which communicates wirelessly with a transceiver (503) in the 
intelligent player (500). 

The EFR file (100) may be delivered into the player (500) in any of a 
number of ways. Similar to the computer system described above in Fig. 4, the 
intelligent player (500) may have a connection (502) to an external network, such 
as the Internet or a cable television system. This connection (502) may be used to 
transmit an EFR file (100) to the player (500). Alternatively, the player (500) may 
include a disc drive (507) which can accept a floppy disc (41 1) or CD-ROM (410) 
bearing an EFR file (100). In yet another alternative, the video disc driver (501) 
may be able to read a CD or CD-ROM and may read an EFR file (1 00) from such 
a disc prior to accepting the optical disc (106) from which the corresponding video 
program is read. 

Fig. 6 illustrates the important principle that the EFR file (100), generated 
on the system (108) illustrated in Fig. 1, can be accepted and used by a number of 
playback platforms (601). Examples of potential playback platforms (601) 
include, but are not limited to, a personal computer with a DVD drive, an Apple® 
computer, an intelligent DVD player, a Sony® Playstation2 or a Microsoft® 
Xbox. 

Fig. 7 is a flowchart detailing the playback software package of the present 
invention that accepts and utilizes an EFR file (100). As noted above, this 
software package may be adapted to run on any playback platform with 
appropriate modification. This adaptation does not affect the package's ability to 
accept and utilize any EFR file (100). This is due, in the preferred embodiment, to 
the fact that the EFR file (100) is an XML file. 

As shown in Fig. 7, the playback platform running the playback software 
package of the present invention (e.g., 408) will display a playlist using the data 
from the EFR (100). The playlist is a hierarchical listing of the divisions of the 
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video program illustrated in Fig, 3. Because this data is stored in the EFR (100), 
the playlist may be reviewed and edited with or without the corresponding optical 
disc (320). 

Preferably, the playlist is displayed using a tree structure as shown in Fig. 
7a. With the tree structure, the user can expand or close any branch of the 
hierarchy that the user wishes to see or work with. As shown in Fig, 7a, the user 
interface of the software package diagramed in Fig. 7 preferably displays the 
playlist (750) as a listing of divisions of the video program on the optical disc. 
Below each listed segment, an indented listing of the subdivisions of that segment 
can be accessed by selecting the icon (753, 754) associated with the listing for that 
segment. 

As shown in Fig. 7a, the listing for the "Main Segment" has a "boxed 
minus" icon associated therewith. This indicates that the listing of subdivisions 
within the "Main Segment," i.e., the "Chapters," is displayed. The user could hide 
the listing of Chapters and subsequent subdivisions by selecting the "boxed minus" 
icon associated with the listing "Main Segment." 

Similarly, the listing for Chap 1 includes a "boxed plus" icon (753), 
indicating that the subdivisions within Chap 1 are not presently listed in the 
playlist (750). These subdivisions of Chap 1 could be shown in an indented list 
under "Chap 1" if the user selects the "boxed plus" icon (753). In contrast, the 
icon (754) associated with the listing for Chap 2 is a "boxed minus" indicating that 
an indented listing of "Subscenes" within Chap 2 is displayed. 

The playlist (750), if partially or wholly expanded to list tjie various 
subdivisions of the video program segments, may exceed the capacity of the 
monitor (751) on which the user interface is being displayed. Consequently, a 
scrolling mechanism for moving through the display playlist on the monitor will 
preferably be provided. Such mechanisms are well-known in the art and would be 
controlled through the user interface of the playback device. 
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The playlist (750) may also include a coding device that advises users when 
and what type of supplemental material is associated with the various subdivisions 
of the video program. In the example shown in Fig. 7a, each listing for a clip may 
include a number of differently colored dots (752). Each color of the dots (752) 
may represent a different type of supplemental information that is associated with 
that clip. For example, a red dot may be included with the listing for any clip for 
which there is a textual plot summary included in the EFR. A blue dot may be 
included with the listing for any clip in which there is potentially objectionable 
subject matter. A green dot may be included in the listing for any clip for which 
there is historical or cultural background information included in the EFR. 

Any type of information associated with a clip may be designated by a 
particular color of dot or any other distinguishing character, symbol, icon, graphic, 
etc. Additionally, the coding system may be applied at any level of the hierarchy 
and not just at the "Clip" level. At any hierarchical level, the elements of the 
coding system (e.g. colored dots) may have associated therewith a number that 
gives an indication of how may subdivisions within that listing include 
supplemental data of that type or how much data of that type is included within 
those subdivisions. 

Returning to Fig. 7, the playback platform running the playback software 
package of the present invention (e.g., 408) will display a playlist using the data 
from the EFR. (701). The user may then select a clip or other division of the 
hierarchy to work with (702). At the clip level, a menu or listing of the 
supplemental information associated with that clip will be displayed (703). The 
user may then explore the additional information associated with the clip (704). 
As noted above, the supplemental information may be of almost any type 
including, but not limited to, cultural notes, linguistic notes, plot summary, 
reviews and critiques, historical or other background notes, and a description of 
potentially objectionable subject matter. (705). 
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In addition to accessing the supplemental information, the user may also 
edit the playlist (706). In editing the playlist, the user may rearrange the order of 
segments or remove segments from the playlist so that those segments are skipped 
during playback of the video program. .The user may also specify that some of the 
supplemental information contained in the EFR be displayed during playback to 
supplement or replace a corresponding video segment. 

When the user has finished editing the playlist (707), the edited playlist is 
stored (71 1). If the user makes no edits to the playlist of the EFR (708) and then 
initiates playback of the unedited playlist (709), the entire video program is played 
(71 0) in the manner recorded on the optical disc. In other words, if the user does 
not edit the playlist and if the initial playlist is complete, a playback of the video 
program is the same as would be the case if the user had no ability to edit the 
playlist and were not using an EFR of the present invention. 

It should also be noted that the user can edit the playlist and store an edited 
playlist using an EFR (100) of the present invention without having possession of 
the corresponding optical disc containing the video program. Portions of the video 
program itself, except perhaps single frames or quotations for indexing purposes, 
are preferably not included in the EFR (100). 

If an edited playlist is produced and stored in the EFR (1 00), and the user 
then initiates playback (712) of the corresponding video program from an optical 
disc using the playback software package diagrammed in Fig. 7, the video program 
is played back subject to any changes in the sequence of segments or omission of 
segments dictated by the edited playlist (713). This may also include displaying a 
textual description of a segment or information regarding that segment included in 
the EFR (100) to supplement or replace the segment itself. 

Consequently, the EFR and playlist of the present invention may be of great 
use to parents and educators who are displaying a video program for children or 
younger age groups and who may consequently wish to remove potentially 
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objectionable content from the playback. This gives the user great flexibility in 
deciding precisely what type of content in the video program is objectionable and 
allows the user to remove only that content. Moreover, the user need not 
experience the content he or she finds objectionable during editing, but can instead 
read a textual description of that content in the information associated with that 
clip or video segment in the EFR (100) of the present invention. 

Fig. 8 illustrates an additional feature of the present invention. The EFR 
(100) may, in addition to the hierarchical divisions of the video program on a 
corresponding optical disc, also include a division of audio segments (310) within 
each clip. Each audio segment (3 1 0) will correspond to a word or phrase within 
the clip (307). The EFR (100) may contain a non-specific description of the 
language or a verbatim listing of the language in each audio segment (310). 
Consequently, any inappropriate word or phrase can be removed from the playlist 
as edited by the user. 

Finally, Fig. 9 illustrates another feature of the playback software package 
of the present invention. As shown in Fig. 9, the user may initiate an analysis of 
an edited playlist (801). The analysis feature of the software will then provide an 
assessment of the edited playlist. The analysis may be on the edited playlist alone 
or on the edited playlist in comparison to the original unedited playlist (802). 

Without a comparison to the original playlist, a number of important 
analyses can be performed on the edited playlist. For example, an indicator of the 
relevance of each video segment to the plot of the video program may be 
associated with that segment in the EFR (100) of the present invention. This 
indicator may be a numeric value or an alphanumeric code. Consequently, the 
analysis feature can assess and quantify damage done to the plot of the program by 
the removal of various video segments from the playlist. Additionally, the 
analysis feature may provide statistics on the edited playlist such as the amount of 
time required for playback. The analysis feature may also indicate the number of 
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segments remaining that include any one or all of the various types of 
supplemental information. For example, the analysis feature may indicate that a 
particular number of the included segments contain indications or descriptions of 
potentially objectionable subject matter and may divide that potentially 
objectionable subject matter in categories and degrees. These statistics are 
generated (806) and then displayed for the user (807). 

If the user desires a comparison to the unedited playlist, the original playlist 
is retrieved (803) and analyzed in comparison with the edited playlist. For 
example, the amount of running time for each playlist, edited and unedited, may be 
compared. These statistics are generated (804) and then displayed for the user 
(805). 

In another aspect of the present invention, if the user has deleted a clip or 
segment from the playlist, and that deleted segment has associated therewith an 
indicator of relevance to plot that is greater than a predetermined threshold, i.e., 
the deleted segment is highly relevant to the plot, the playback software package 
of the present invention may automatically display a textual description of the 
deleted segment in place of the segment to prevent the gap in plot that would 
otherwise occur in playback due to the deletion of the segment. Additionally, it 
should be understood that at any point in time during playback, using the playback 
software package of the present invention, the user can pause the playback and 
access the supplemental information associated with that or another segment of the 
video program. 

Finally, using the playback software package of the present invention, the 
user may designate one or more types of supplemental information that are of 
particular interest in the purpose for which the user is playing back the video 
program. The playback software may then advise the user during playback when a 
particular segment of the video program has supplemental information of the 
designated type or types associated therewith. The user may then pause the 
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playback and access the information or instruct the playback software to 
automatically pause at such a point and display the relevant supplemental 
information. 

This feature may also be tied to an indicator as to the significance of the 
supplemental information such that only supplemental information of the 
designated type which is coded as having a particular level of significance is 
advertised to the user during playback. This can minimize disruption to the 
playback if so desired by the user. 

The preceding description has been presented only to illustrate and describe 
the invention. It is not intended to be exhaustive or to limit the invention to any 
precise form disclosed. Many modifications and variations are possible in light of 
the above teaching. 

The terms "playback platform" are "video playback device" are used herein 
to refer generally to those devices, such as the computer (408) and intelligent 
video player (500), that run the playback software package of the present invention 
and control an optical disc drive to playback a recorded video program in 
accordance with an edited or unedited playlist of an EFR of the present invention. 

The preferred embodiment was chosen and described in order to best 
explain the principles of the invention and its practical application. The preceding 
description is intended to enable others skilled in the art to best utilize the 
invention in various embodiments and with various modifications as are suited to 
the particular use contemplated. It is intended that the scope of the invention be 
defined by the following claims. 
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WHAT IS CLAIMED IS: 

1 . A video playback system for piaying back a video program recorded 
on a recording medium, the system comprising: 
a video playback device; and 

an recording medium reader for reading a recording medium on which said 
video program is recorded, said reader being controlled by said video playback 
device, 

wherein said video playback device controls playback of said video 
program in accordance with a user-editable electronic data file existing in said 
video playback device independent of data recorded on said recording medium; 
and 

wherein said electronic data file includes a user-editable playlist that 
divides said video program as recorded on said recording medium into at least one 
hierarchy of video segments. 

2. The system of claim 1 , wherein said electronic data file further 
comprises supplemental information associated with at least one of said segments 
of said video program. 

3 . The system of claim 2, wherein said supplemental information 
includes an alphanumeric code that rates a characteristic of said associated 
segment. 

4. The system of claim 3, wherein said playback device generates a 
statistical analysis of characteristics of said video program as played back in 
accordance with said edited playlist based on said alphanumeric code. 
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5. The system of claim 2, wherein: 

said system further comprises a user interface including a display device on 
which said playlist is displayed; and 

said displayed playlist comprises a listing of said segments of said video 
program and a coding system that indicates what types of said supplemental 
information are associated with each said segment. 

6. The system of claim 2, wherein said system further comprises a user 
interface, where said playback device accesses and displays said supplemental 
information under control of a user operating said user interface. 

7. The system of claim 1 , wherein said playback device further 
comprises a user interface and records changes to said playlist made by said user 
through said user interface. 

8. The system of claim 7, wherein: 

said electronic data file further comprises a description of each of a 
plurality of audio segments of said video program; and 

said playback device records deletions of audio segments from said playlist 
made by said user through said user interface. 

9. The system of claim 7, wherein said playback device generates a 
statistical analysis of characteristics of said video program as played back in 
accordance with said edited playlist. 

1 0. A method of controlling and enhancing playback of a video program 
recorded on a recording medium, said method comprising controlling playback of 
said video program with a video playback device in accordance with a user- 
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editable electronic data file existing in said video playback device independent of 
data recorded on said recording medium, wherein said electronic data file includes 
a user-editable playlist that specifies a hierarchical division of segments of said 
video program as recorded on said recording medium. 

1 1 . The method of claim 10, further comprising enhancing playback of 
said video program by associating supplemental information in said electronic data 
file with at least one of said specified segments of said video program. 

12. The method of claim 1 1 , further comprising 

displaying said playlist comprising a listing of said segments of said video 
program; and 

displaying a coding system that indicates what types of said supplemental 
information are associated with each said segment. 

1 3 . The method of claim 1 1 , further comprising accessing and 
displaying said supplemental information under control of a user operating a user 
interface of said video playback device. 

14. The method of claim 10, further comprising recording an edited 
playlist in said electronic data file, said edited playlist including changes to said 
playlist made by said user through a user interface of said video playback device. 

15. The method of claim 14, further comprising comparing said playlist 
to said edited playlist to assess an impact of said changes made to said playlist by 
said user. 
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16. The method of claim 14, further comprising analyzing said edited 
playlist to determine characteristics of said edited playlist 

17. A method of providing an electronic data file for use in controlling 
playback of a corresponding video program recorded on a recording medium, said 
method comprising: 

dividing said video program as recorded on said recording medium into a 
hierarchy of segments; and 

storing said hierarchy in an electronic data file independent of data 
recorded on said recording medium, 

18. The method of claim 1 7, further comprising associating 
supplemental information with one or more of said segments, said supplemental 
information being recorded in said electronic data file. 

19. A method of doing business in which customers are provided with a 
means for controlling and supplementing playback of a video program recorded on 
a recording medium, said method comprising selling an electronic data file that 
corresponds to said video program on said optical disc, wherein said electronic 
data file includes a user-editable playlist that divides said video program into video 
segments and includes supplemental information associated with one or more of 
said video segments, said data file being usable to control playback of said video 
program on a video playback device. 

20. The method of claim 1 9, wherein said playlist divides skid video 
program into a hierarchy of video segments. 
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2 1 . The method of claim 1 9, further comprising delivering said 
electronic data file to a customer via the Internet 



25 



WO 02/062061 



PCTYUSO 1/47062 




1/11 



WO 02/062061 



PCT/US01/47062 



Q Start J 



Read audiovisual data 
from recording medium 
(201) 



Display audiovisual 
program 
(202) 




YES 



Record clip parameters 
(204) 



NO 




YES 



Record additional data 
with clip parameters 
(206) 




YES 



Fig. 2 



Generate EFR file 




(208) 


H 



^ End ^ 



2/11 



WO 02/062061 PCT/US01/47062 



320 




Preview 
(302) 



330 



Main 
segment 
(the film) 

(301) 



Chap. 
1 

(304-1) 



Chap. 
2 

(304-2) 



Chap. 
3 

(304-3) 



Trailer, 
Out- 
takes, 
Making 
of, etc. 
(303) 



Chap. 
N 

(304-N) 



331 



Scene 
1 

(305-1) 



Scene 
2 

(305-2) 



Scene 
3 

(305-3) 



Scene 
n 

(305-n) 



Sub- 
scene 
1 

(306-1) 



Sub- 
scene 
2 

(306-2) 



Clip 
1 

(307-1) 

Clip 
2 

(307-2) 

Clip 
3 

(307-3) 



Sub- 
scene 
3 

(306-3) 



Clip 
m 

(307-m) 



Sub- 
scene 
M 

(306-M) 



Fig. 3 



3/11 



WO 02/062061 



PCT/US01/47062 




Scene 
2 

(305-2) 



Scene 
3 

(305-3) 



Fig. 3a 



Scene 
n 

(305-n) 



Sub- 
scene 
1 

(306-1) 



Sub- 
scene 
2 

(306-2) 



Sub- 
scene 
3 

(306-3) 



Clip 
1 

(307-1) 

Clip 
2 

(307-2) 

Clip 
3 

(307-3) 



Clip 
m 

(307-m) 



Sub- 
scene 
M 

(306-M) 



4/11 



WO 02/062061 



PCT7US01/47062 




5/11 



WO 02/062061 



PCT/US01/47062 




6/11 



WO 02/062061 



PCT/US01/47062 



108 




Intelligent Video 




Intelligent Video 




Intelligent Video 


Disc Player A 




Disc Player B 




Disc Player C 


(601a) 




(601b) 




(601c) 



Fig. 6 



7/11 



WO 02/062061 



PCT/USO 1/4 7062 



Entire video program is 
played 
(710) 



Store edited playlist 
(711) 




8/11 



WO 02/062061 



PC17US01/47062 



EFR Playlist 








TO Prp\/ip\A/ 
uj ncvicvv 








□ iviain oegmeni 








rri fhan 4 








753 H Phan 9 








~-l 1 1 O Ann «i 

cn ocene i 








ocene 4: 




752 




754 <^^^ is bubscene 1 








a oUDScene ^. 










0 r\ C\ c\ 1 
U U U \j / 








on / 
u u / 






Lrlip O 


r% 0 rs I \ 
V V u / I 






Lflip *f 


0 / 1 

0 / I 






V/lip 0 


r\ 0 0 
U U U U U 






irk « 


U U 






! L/llp / 


AAA 

u u 0 






to ouDscene 0 








en ocene 0 








tn bcene4 








cd unap 0 








EB Chap 4 








1 EB Chap 5 








S Chap 6 






EB Chap 7 








63 Chap 8 








63 Chap 9 








EB Chap 10 








EB Chap 11 








EB Chap 12 








B Trailer 








El Out-takes 








ffl Director's notes 









751 



750 



Fig. 7a 



9/11 



WO 02/062061 



PCT/US01/47062 



Fig. 8 



Clip 
(307) 



Audio 
segment 
1 

(310-1) 



Audio 
segment 
2 

(310-2) 



Audio 
segment 
3 

(310-3) 




10/11 



PC1YUS01/47062 




YES- 



Generate edit 
statistics 
(806) 



Display edit statistics, 
e.g., damage to plot, 
for user 
(807) 



Retrieve unedited 
playlist and compare 
(803) 


} 




Generate comparison 
statistics 
(804) 


} 


f 


Display comparison 

statistics, e.g., 
damage to plot, for 
user 
(805) 


\< 





Fig. 9 



11/11 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US01/47062 



A. CLASSIFICATION OF SUBJECT MATTER 
IPC(7) : H04N/593 
US CL : 386/55 

According to International Patent Classification (IPC) or to both national classification and IPC 



FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 386/55, 70, 75 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category * 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



X,P 



US 6,289,165 Bl (ABECASSIS) 11 September 2001 (11.09.2001), figure 6A; figure 7A; 
column 2, lines 46-52; column 16, lines 1-3; column 12, line 32; column 15, lines 63-65; 
column 16, lines 28-39; column 20, lines 4-16; column 21, lines 3-19; column 24, lines 
26-36; column 28, lines 1-2; column 39, lines 21-25. 



1-21 



1 1 Further documents are listed in the continuation of Box C. | | See patent family annex. 



"L" 



•O" 
"P" 



Spccial categories of cited documents: 

document defining the general state of the an which is not considered to be 
of particular relevance 

earlier application or patent published on or after the international filing date 

document which may throw doubts on priority claim(s) or which is cited to 
establish the publication date of another citation or other special reason (as 
specified) 

document referring to an oral disclosure, use, exhibition or other means 



t published prior to the international filing date but later than the 
priority date claimed 



later document published after the international filing date or priority 
date and not in conflict with the application but cited to understand the 
principle or theory underlying the invention 

document of particular relevance; the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive step 
when the document is token alone 

document of particular relevance; the claimed invention cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

document member of the same patent family 



Date of the actual completion of the international search 
30 May 2002 (30.05.2002) 



Date of mailing of the international search report 



27 Ml 



Name and mailing address of the ISA/US 

Commissioner of Patents and Trademarks 
BoxPCT 

Washington, D.C. 20231 
Facsimile No. (703)305-3230 



Mj^j^zed^J^cer 
Andrew Chris tensen 
Telephone No. (703)305-3900 



Form PCT/ISA/210 (second sheet) (July 1998) 



System and method for sending and receiving a video as a slide show over a 
Computer network 



Publication number: 
Publication date: 
Inventor: 

Applicant: 
Classification: 

- international: 

- European: 
Application number: 
Priority number(s): 



AU6672298 
1998-09-09 

COLBY KENNETH W; KENNER BRIAN; WEATHERSBY 
GUY P; BROWNELL LONNIE J; FLYNN PETER K 

INTERVU INC 

H04N7/26; H04N7/52; H04N7/26; H04N7/52; (IPC1-7): 
H04N7/26 

H04N7/2!6E6i HS4N7/26T; W04N7/52A 
AU19980066722D 19980225 

US1 9970806093 19970225; WO1998US03904 
19980225 



Also published as: 

13 W09837699(A1) 



Report a data error here 



Abstract not available for AU6672298 
Abstract of corresponding document: W09837699 
A system and method for encoding and decoding 
digitized audio/video files prepares a slide show 
of still images and a low bit rate audio stream 
which can be downloaded in real time over a 
typical connection to a computer network. The 
quality of audio/video file is subsequently 
improved by downloading in successive passes 
the remaining video frames, which are restored to 
their original order, and the original high-quality 
audio content. 



SERVER/PS! VIDEO PUMP 122 



TRANSFER J*— 
MONITOR 



134 ■ 



m 

RAV PILES t 



11? 



s 
m 



V 



DOOTENT MANAGER f W 



FRAME 

SELECTOR 


HTRAXSOO0ER] 








1223112213 




Data supplied from the esp@cenet database - Worldwide 



